Real-time FEM based control of soft surgical robots

International audienceIn this paper, we present a new method for the control of soft surgical robots based on the real-time inverse simulation with internal deformation computed through the use of Finite Element Method. We also consider the coupling of this method with a modified version of the same algorithm for parametrization of soft-tissue models, in order to control the navigation of the robot while gathering information on the surrounding organs

Real-time simulation of hydraulic components for interactive control of soft robots

International audienceIn this work we propose a new method for online motion planning in the task-space for hydraulic actuated soft robots. Our solution relies on the interactive resolution of an inverse kinematics problem, that takes into account the properties (mass, stiffness) of the deformable material used to build the robot. An accurate modeling of the mechanical behavior of hydraulic components is based on a novel GPU parallel method for the real-time computation of fluid weight distribution. The efficiency of the method is further increased by a novel GPU parallel leveraging mechanism. Our complete solution has been integrated within the open-source SOFA framework. In our results, we validate our simulation with a fabricated silicone cylinder and we demonstrate the usage of our approach for direct control of hydraulic soft robots

Introducing interactive inverse FEM simulation and its application for adaptive radiotherapy

International audienceWe introduce a new methodology for semi-automatic deformable registration of anatomical structures, using interactive inverse simulations. The method relies on non-linear real-time Finite Element Method (FEM) within a constraint-based framework. Given a set of few registered points provided by the user, a real-time optimization adapts the boundary conditions and(/or) some parameters of the FEM in order to obtain the adequate geometrical deformations. To dramatically fasten the process, the method relies on a projection of the model in the space of the optimization variables. In this reduced space, a quadratic programming problem is formulated and solved very quickly. The method is validated with numerical examples for retrieving Young's modulus and some pressures on the boundaries. Then, we apply the approach for the registration of the parotid glands during the radiotherapy of the head and neck cancer. Radiotherapy treatment induces weight loss that modifi es the shape and the positions of these structures and they eventually intersect the target volume. We show how we could adapt the planning to limit the radiation of these glands.Nous introduisons une nouvelle méthode de recalage déformable semi-automatique de structures anatomiques, à l'aide de simulations inverses interactives. La méthode est basée sur la méthode des éléments finis(FEM) et revient à résoudre un système de contraintes. Etant donné un ensemble de quelques points fournies par l'utilisateur, une optimisation en temps réel adapte les conditions aux limites et(/ou) des paramètres de la FEM dans le but d'obtenir les déformations géométriques adéquates. Pour accélérer les calculs de manière conséquente, la méthode repose sur une projection du modèle dans l'espace des variables d'optimisation. Dans cet espace réduit, un problème de programmation quadratique est formulé et résolu très rapidement. La méthode est validée par des exemples numériques (récupérer le module de Young et des pressions à appliquer sur le modèle). Ensuite, nous appliquons l'approche pour le recalage des glandes parotides pendant la radiothérapie de la tête et du cou. Un traitement de radiothérapie induit généralement une perte de poids chez le patient qui modifie la forme et la position de ces structures. Structures qui finissent par entrer dans le volume cible. Nous montrons comment nous pourrions adapter la planification afin de limiter le rayonnement de ces glandes

Vascular neurosurgery simulation with bimanual haptic feedback

International audienceVirtual surgical simulators face many computational challenges: they need to provide biophysical accuracy, realistic feed-backs and high-rate responses. Better biophysical accuracy and more realistic feed-backs (be they visual, haptic.. .) induce more computational footprint. State-of-the-art approaches use high-performance hardware or find an acceptable trade-off between performance and accuracy to deliver interactive yet pedagogically relevant simulators. In this paper, we propose an interactive vascular neurosurgery simulator that provides bi-manual interaction with haptic feedback. The simulator is an original combination of states-of-the-art techniques that allows visual realism, bio-physical realism, complex interactions with the anatomical structures and the instruments and haptic feedback. Training exercises are also proposed to learn and to perform the different steps of intracranial aneurysm surgery (IAS). We assess the performance of our simulator with quantitative performance benchmarks and qualitative assessments of junior and senior clinicians

FEM-based Deformation Control for Dexterous Manipulation of 3D Soft Objects

International audienceIn this paper, a method for dexterous manipulation of 3D soft objects for real-time deformation control is presented, relying on Finite Element modelling. The goal is to generate proper forces on the fingertips of an anthropomor-phic device during in-hand manipulation to produce desired displacements of selected control points on the object. The desired motions of the fingers are computed in real-time as an inverse solution of a Finite Element Method (FEM), the forces applied by the fingertips at the contact points being modelled by Lagrange multipliers. The elasticity parameters of the model are preliminarly estimated using a vision system and a force sensor. Experimental results are shown with an underactuated anthropomorphic hand that performs a manipulation task on a soft cylindrical object

Automated planning for robotic guidewire navigation in the coronary arteries

International audienceSoft continuum robots, and comparable instruments allow to perform some surgical procedures noninvasively. While safer, less morbid and more cost-effective, these medical interventions increase the complexity for the practitioners: the manipulation of anatomical structures is indirect through telescopic and flexible devices and the visual feedback is indirect through monitors. Interventional cardiology is an example of complex procedures where catheters and guidewires are manipulated to reach and treat remote areas of the vascular network. Such interventions may be assisted with a robot that will operate the tools but the planning (choice of tools and trajectories) remains a complex task. In this paper we use a simulation framework for flexible devices inside the vasculature and we propose a method to automatically control these devices to reach specific locations. Experiments performed on 15 patient geometries exhibit good performance. Automatic manipulation reaches the goal in more than 90% of the cases

Software toolkit for modeling, simulation and control of soft robots

International audienceThe technological differences between traditional robotics and soft robotics have an impact on all of the modeling tools generally in use, including direct kinematics and inverse models, Jacobians, and dynamics. Due to the lack of precise modeling and control methods for soft robots, the promising concepts of using such design for complex applications (medicine, assistance, domestic robotics...) cannot be practically implemented. This paper presents a first unified software framework dedicated to modeling, simulation and control of soft robots. The framework relies on continuum mechanics for modeling the robotic parts and boundary conditions like actuators or contacts using a unified representation based on Lagrange multipliers. It enables the digital robot to be simulated in its environment using a direct model. The model can also be inverted online using an optimization-based method which allows to control the physical robots in the task space. To demonstrate the effectiveness of the approach, we present various soft robots scenarios including ones where the robot is interacting with its environment. The software has been built on top of SOFA, an open-source framework for deformable online simulation and is available at https://project.inria.fr/softrobot

Modèle inverse de robots souples basé sur des méthodes d’optimisation, avec prise en compte des contacts

La robotique souple s’inspire de la nature, de la manière dont les organismes vivants se déplacent et adaptent leur forme à leur environnement. Contrairement aux robots traditionnels, les robots souples accomplissent des tâches avec plus de flexibilité. Les matériaux souples avec lesquels ils sont construits les rendent plus sûrs pour des environnements fragiles.Cependant, le domaine de la robotique souple pose de nouveaux défis, en particulier pour la modélisation et le contrôle. Dans cette thèse, nous visons à fournir des méthodes génériques pour leur modélisation. Les méthodes sont basées sur la méthode des éléments finis pour capturer les déformations de la structure du robot, et de son environnement, quand il est déformable. Nous formulons le problème de leur cinématique inverse et dynamique inverse comme un programme d’optimisation, permettant de gérer facilement des contraintes aux actionneurs et des problèmes de singularité. Nous sommes en mesure de contrôler plusieurs types d’actionnement, tels que les actionnements par câbles, pneumatiques et hydrauliques.De plus, la plupart des applications impliquent une interaction du robot avec des obstacles. Or, la cinématique des robots souples dépend fortement des facteurs environnementaux. Nous proposons ainsi de nouvelles méthodes qui prennent en compte les contacts dans le processus d’optimisation. Enfin, nous proposons de contrôler certaines tâches de locomotion et de préhension nécessitant l’utilisation de contacts frottants (statique). Nous accordons une attention particulière à fournir des solutions avec des performances temps réel, permettant un contrôle en ligne des robots dans des environnements changeant.Soft robotics draws its inspiration from nature, from the way living organisms move and adapt their shape to their environment. In opposition to traditional rigid robots, soft robots are built from highly compliant materials, allowing them to accomplish tasks with more flexibility. They are safer when working in fragile environment, which allows for potential use of soft robotics in the fields of manufacturing and medicine.Yet, the field of soft robotics brings new challenges, in particular for modeling and control. Within this thesis we aim at providing generic methods for soft robot modeling, without assumptions on the geometry. The methods are based on the finite element method to capture the deformations of the robot’s structure and of its environment when deformable. We formulate the problem of their inverse kinematics and dynamics as optimization programs, allowing easy handling of constraints on actuation and singularity problems. We are able to control several types of actuation, such as cable, pneumatic and hydraulic actuations.Moreover, most of the applications involve interaction of the robot with obstacles. Yet soft robots kinematics is highly dependent on environmental factors. We propose new methods that include contacts into the optimization process. These methods make an important step as we think that the knowledge of contacts in the modeling is all the more important. Finally, we propose to control some soft robots during locomotion and grasping tasks which require the use of contact with static friction. We give a particular attention to provide solutions with real-time performance, allowing online control in evolving environments

Modèle inverse des robots souples basé sur l'optimisation, avec gestion des contacts

La robotique souple s'inspire de la nature, de la façon dont les organismes vivants se déplacent et adaptent leur forme à leur environnement. Contrairement aux robots rigides traditionnels, les robots souples sont construits à partir de matériaux hautement compliants, leur permettant d'accomplir des tâches avec plus de flexibilité et d'adaptabilité. Ils sont plus sûrs lorsque l'on travaille dans un environnement fragile. Ils ont l'avantage de produire des forces faibles adaptées à la manipulation / interaction avec des objets / environnements sensibles. Ces caractéristiques permettent une utilisation potentielle de la robotique souple dans les domaines industriels et la médecine.Mais le domaine de la robotique douce pose de nouveaux défis, notamment pour la modélisation et le contrôle. Dans cette thèse, nous visons à fournir des méthodes génériques pour la modélisation de robots souples, sans hypothèses sur la géométrie. Les méthodes sont basées sur la méthode des éléments finis pour capturer les déformations de la structure du robot et de son environnement lorsqu'il est déformable. Nous formulons le problème de leur cinématique et dynamique inverses sous forme de programmes d'optimisation, permettant une gestion aisée des contraintes sur les problèmes d'actionnement et de singularité. Nous sommes en mesure de contrôler plusieurs types d'actionnements, tels que les actionnements par câble, pneumatiques et hydrauliques.De plus, la plupart des applications impliquent une interaction du robot avec des obstacles. Pourtant, la cinématique des robots souples dépend fortement des facteurs environnementaux. Nous proposons de nouvelles méthodes qui incluent les contacts dans le processus d'optimisation. Ces méthodes constituent une étape importante car nous pensons que la connaissance des contacts dans la modélisation est d'autant plus importante. Enfin, nous proposons de contrôler certains robots souples lors de tâches de locomotion et de préhension qui nécessitent l'utilisation du contact avec frottement statique. Nous accordons une attention particulière à fournir des solutions aux performances en temps réel, permettant un contrôle en ligne dans des environnements évolutifs.Soft robotics draws its inspiration from nature, from the way living organisms move and adapt their shape to their environment. In opposition to traditional rigid robots, soft robots are built from highly compliant materials, allowing them to accomplish tasks with more flexibility and adaptability. They are safer when working in fragile environment. They have the advantages of pro- ducing low forces that are suitable for manipulating/interacting with sensitive objects/surroundings without harming them. These characteristics allow for potential use of soft robotics in the fields of manufacturing and medicine.But the field of soft robotics brings new challenges, in particular for modeling and control. Within this thesis we aim at providing generic methods for soft robot modeling, without assumptions on the geometry. The methods are based on the finite element method to capture the deformations of the robot’s structure and of its environment when deformable. We formulate the problem of their inverse kinematics and dynamics as optimization programs, allowing easy handling of constraints on actuation and singularity problems. We are able to control several types of actuation, such as cable, pneumatic and hydraulic actuations.Moreover, most of the applications involve interaction of the robot with obsta- cles. Yet soft robots kinematics is highly dependent on environmental factors. We propose new methods that include contacts into the optimization process. These methods make an important step as we think that the knowledge of con- tacts in the modeling is all the more important. Finally, we propose to control some soft robots during locomotion and grasping tasks which require the use of contact with static friction. We give a particular attention to provide solutions with real-time performance, allowing online control in evolving environments

Soft robots locomotion and manipulation control using FEM simulation and quadratic programming

International audienceIn this paper, we propose a method to control the motion of soft robots able to manipulate objects or roll from one place to another. We use the Finite Element Method (FEM) to simulate the deformations of the soft robot, its actuators, and surroundings when deformable. To find the inverse model of the robot interacting with obstacles, and with constraints on its actuators, we write the problem as a quadratic program with complementarity constraints. The novelty of this work is that friction contacts (sticking contact only) are taken into account in the optimization process, allowing the control of these specific tasks that are locomotion and manipulation. We propose a formulation that simplifies the optimization problem, together with a dedicated solver. The algorithm has real-time performance and handles evolving environments as long as we know them. To show the effectiveness of the method, we present several numerical examples, and a demonstration on a real robot